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Asymptotic fluctuation theorems are statements of a Gallavotti-Cohen symmetry in the 

O 

rate function of either the time-averaged entropy production or heat dissipation of a process. 
Such theorems have been proved for various general classes of continuous-time deterministic 
and stochastic processes, but always under the assumption that the forces driving the system 
are time independent, and often relying on the existence of a limiting crgodic distribution. In 

i— i this paper we extend the asymptotic fluctuation theorem for the first time to inhomogeneous 

continuous-time processes without a stationary distribution, considering specifically a finite 
state Markov chain driven by periodic transition rates. We find that for both entropy 
production and heat dissipation, the usual Gallavotti-Cohen symmetry of the rate function 

i ^ i is generalized to an analogous relation between the rate functions of the original process 

and its corresponding backward process, in which the trajectory and the driving protocol 
have been time-reversed. The effect is that spontaneous positive fluctuations in the long 
time average of each quantity in the forward process are exponentially more likely than 

o 

spontaneous negative fluctuations in the backward process, and vice-versa, revealing that 
the distributions of fluctuations in universes in which time moves forward and backward 

O 

ON are related. As an additional result, the asymptotic time-averaged entropy production is 



o 

> 

X 



obtained as the integral of a periodic entropy production rate that generalizes the constant 
rate pertaining to homogeneous dynamics. 



I. INTRODUCTION 



Following the initial computer simulations of Evans et. al. [8] and the pioneering paper by 
Gallavotti and Cohen [12], the study of fluctuation theorems has led to a fascinating confluence 
of irreversible thermodynamics, stochastic processes and large deviation theory. Unlike its orig- 
inal role in statistical physics of formalizing the thermodynamic limit for equilibrium ensembles, 
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the contribution large deviation theory makes in this context is to characterize the fluctuations in 
the long time average of the entropy production or heat dissipation of a stochastic process, which 
models a physical system whose number of degrees of freedom or incomplete description makes a de- 
terministic treatment infeasible. To see how this characterization comes about and these disparate 
fields fit together, consider an stochastic process £t over a state space S with law fj,(x, t), which for 
simplicity we take to be either a density or discrete distribution. The Gibbs entropy of the pro- 
cess, viewed as an ensemble of paths (or a measure over that ensemble), is — J s log fj,(x, t)fi(dx, t), 
which leads one to identify the entropy along a single stochastic trajectory as — log/i(^,t) [33 . 
The time derivative of this quantity equals the difference between the rate of entropy produced by 
the stochastic particle and the rate of entropy, or heat (divided by a nonphysical temperature), 
dissipated to its environment. Maes [25] as well as Lebowitz and Spohn [23] further recognized that 
under general circumstances the total entropy production equals the logarithmic Radon-Nikodym 
derivative of the forward path measure P governing the process with respect to its corresponding 
backward path measure P B which, under Crooks' more general definition [4j, is obtained by time- 
reversing all temporal inhomogeneities driving the process and composing with a path-reversal 
transformation. As P B (co) equals the probability of observing a path uj unfold in reverse as time 
runs backward from time t to 0, the entropy production may be interpreted as the log likelihood 
of observing w in a universe in which time runs forward as opposed to backward. The existence of 
the derivative (i.e, the equivalence of P and P B ) depends on a condition called at times dynamic 
reversibility [27] or ergodic consistency [9], which ensures that the time-reversal of any trajectory 
realizable in the forward process is realizable in the backward process. 

The Radon-Nikodym definition for entropy production was later justified thoroughly by Maes 
and collaborators for a wide range of deterministic and stochastic processes |26| [27] . It is nearly 
identical to that of the dissipation function Q in deterministic mechanics [9] [33] , except that the 
forward and backward measures in that case put full mass on the constant energy manifold of 
trajectories obeying Hamilton's equations. The logarithmic derivative is nonzero for nonstationary 
processes, which model physical systems evolving far from equilibrium, as well as for stationary 
ones violating detailed balance, modeling systems in a nonequilibrium steady state. Its expectation 
under P equals the relative entropy of P with respect to P B , which is always nonnegative, consistent 
with a weak reading of the second law of thermodynamics. The time-extensive microscopic heat 
dissipation Lebowitz and Spohn termed an action functional, a quantity that is equal to the above 
logarithmic derivative up to the difference of boundary terms — log > precisely the net change 
in system entropy. 



3 



Recent work has exploited the thermodynamic framework above by proving that either the 
time-averaged entropy production or heat dissipation satisfies a large deviation principle, whose 
corresponding rate function satisfies the same symmetry as the one proved by Gallavotti and 
Cohen [12] to hold for the time-averaged phase space contraction of chaotic dynamical systems. 
We refer to this as an asymptotic fluctuation theorem (AFT) or, simply, a fluctuation theorem, 
and distinguish it from transient fluctuation theorems, which hold at finite times but are not 
large deviation results. Kurchan first proved an AFT [21] for the entropy production of Langevin 
processes under the assumption of nondegeneracy for the maximum eigenvalue of their evolution 
operator, and Lebowitz and Spohn then proved it [24 for the heat dissipation of time-homogeneous 
Markov chains and Ito diffusions, whose assumed limiting stationary distribution guarantees this 
condition. The cases of continuous and discrete time Markov chains were handled in a rigorous 
fashion by Jiang et. al. [18] [19], and Ge et. al. extended the discrete time case to include 
time-periodic inhomogeneities in the transition matrix [16]. Maes took a different approach from 
the others, studying the dynamics of finite volume Gibbs states on Z d under general space-time 
potentials [25]. 

Interestingly, it has been shown that unlike their entropy production, the heat dissipation of 
Langevin processes does not satisfy an AFT, at least in the conventional sense |39 |, 140] . displaying 
importance of the boundary terms distinguishing these quantities. Along similar lines, Rakos 
and Harris have recently shown [30 that infinite state spaces can result in the divergence of the 
boundary terms, causing a breakdown of the Gallavotti-Cohen symmetry. It is unclear, however, 
whether this breakdown, which arises from the failure of the Hamiltonian operator to satisfy 
Kurchan's nondegeneracy requirement, is really a function of the state space's cardinality as much 
as its non-compactness (the state space of a Markov chain being effectively endowed with the 
discrete metric). In the case of deterministic dynamics, Bonetto et. al. have argued that apparent 
violations of AFTs by particle systems with singularities in the interparticle potential (owing, for 
example, to hard cores) can be corrected by subtracting "unphysical" singular terms from the 
phase space contraction rate [2] . 

While so much effort recently has gone into circumscribing the range of applicability of the 
AFT, the purpose of our present work is to extend it for the first time to the case of inhomoge- 
neous, continuous time dynamics. We consider in particular a Markov chain on a finite state space, 
whose infinitesimal generator is a continuous and periodic function of time but only required to be 
irreducible at a single moment. The finiteness of the state space ensures that the complexity of the 
model is isolated within the time dimension, avoiding in particular the issues raised in Ref. [30 . 
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Such a process can be used to model phenomena as diverse as the fluctuation-driven transport of 
molecular motors [1, stochastic resonance in lasers and neuron firing [13], quasienergy banding in 
periodically-driven mesoscopic electric circuits [3J, and seasonality in population dynamics |31j . as 
well as periodically-driven deterministic processes amenable to coarse-graining. Continuous time 
models of all of these phenomena were previously outside the scope of AFTs, all of which had 
been proved under the assumption of homogeneous dynamics, because they rely fundamentally 
on a time-dependent protocol driving the process. Indeed, few systems in nature operate within 
a static environment, and so to gain true scientific relevance AFTs must ultimately accommo- 
date time-inhomogeneities. While our assumption of periodicity remains a restriction to potential 
applications, we believe that having laid out in this paper the mathematical issues involved in 
introducing time-dependent driving, our arguments can serve as a blueprint for future work that 
seeks to loosen this restriction. 

Whereas the authors mentioned above have proved AFTs for either the action functional or 
entropy production of a process alone, we prove for both, finding that the absence of boundary 
terms in the action functional simplifies the derivation of its free energy but complicates that of its 
fluctuation symmetry. The fluctuation symmetry for both quantities takes the form I(z)—I B (—z) = 
z, where / is the rate function under the forward process and I B under the backward process, which 
reduces to the usual Gallavotti-Cohen symmetry when the driving protocol is symmetric inside 
every driving period and, hence, has no temporal orientation. Its interpretation is that spontaneous 
positive fluctuations in the long time average of each quantity in one process is exponentially more 
likely than spontaneous negative fluctuations in the other, a relation that is symmetric with respect 
to the two processes. 

We also derive the almost sure asymptotic time-averaged entropy production as the integral 
Jq e p (s)ds, where e p (s) is a T-periodic instantaneous entropy production rate, with T £ M + the 
period of the driving. This expression generalizes existing ones known for homogeneous |14[ [TT?] and 
periodically inhomogeneous, but discrete time [16], chains. Our proofs are guided strategically by 
those of Lebowitz and Spohn, but, as discussed above, the inhomogeneity of the process necessitates 
more involved, and rigorous, arguments. The virtue of this is that our proofs, unlike those of 
Leibowitz and Spohn, do not rely explicitly on the existence of a stationary distribution for the 
process (indeed, only a periodic quasi-stationary distribution exists), raising the open question of 
how insensitive the existence of an AFT is to the details of the asymptotic regime of the process. 
Put another way, how far can our assumption of periodicity on the driving rates be relaxed but still 
guarentee an AFT? We address this question in the final section of the paper, proving that uniform 



5 



continuity and boundedness of the rates alone, which are implied by the assumptions described in 
section II, are not sufficient. This shows that any minimal set of conditions on the rates are closer 
to those assumed in this paper than one might initially suspect. 

A number of theoretical and experimental results already exist for periodically-driven stochastic 
processes. Integral and transient fluctuation theorems have been proved for periodically-driven 
two-state Markov chains [32, 37 , quantum systems [22 , as well as classical harmonic oscillators 
modeled by a Langevin process [20, 35 . It must be emphasized that these studies differ from 
ours because they do not prove large deviations results. As the technicalities we deal with in this 
paper reveal, the transition from transient to asymptotic fluctuation theorem, even for processes 
as simple as finite state Markov chains, is not an automatic or obvious one, but depends on the 
details of the driving protocol. 

Similarly, in spite of the close relation between discrete and continuous time Markov chains, our 
results cannot be obtained from those of Ge. et. al. The set of transition probabilities in their case 
is finite, and the uniformity of the waiting times between jumps enables the inhomogeneous chain 
on S to be re-represented as a homogeneous chain on the larger (but still finite) space S T , where 
X" £ N is the period of the transition probabilities. Our set of transition rates is uncountable, 
on the contrary, and is sampled randomly by the process, ensuring no convenient reduction to 
a homogeneous problem. It is possible, of course, that with the proper scaling of the transition 
probabilities (under which the periodicity T" would diverge), the continuous time path measure 
may be obtained as the weak limit of the discrete time path measures, analogous to Donsker's 
theorem for Brownian motion. However, as has been mentioned, the rate functions for the heat 
dissipation and entropy production of the continuous process do not simply inherit the fluctuation 
symmetry of their discrete approximants. Instead, the internal symmetry of each rate function is 
replaced by a symmetry between it and its counterpart under the corresponding backward process. 

The rest of the paper is organized as follows. We begin by making the appropriate definitions, 
and then derive a backward equation for the moment generating function of the action functional, 
whose fundamental solution we obtain using both Floquet theory and the time-ordered exponential 
operator. This enables us to identify the free energy of the action functional as the nondegenerate 
principal Floquet eigenvalue of the fundamental solution, thereby proving the existence and strict 
convexity of its Legendre-Fenchel transform, the rate function. We then show that the free energies 
of the action functional under the forward and backward path measures satisfy a symmetry relation 
analogous to the one found by Lebowitz and Spohn in the case of time-homogeneous dynamics, 
implying the fluctuation theorem symmetry between the forward and backward rate functions. All 
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of these results are subsequently proved for the entropy production in place of the action functional, 
along with the consequent generalization of the second law of thermodynamics and a derivation 
of the asymptotic time-averaged entropy production and associated production rate. We conclude 
with a look beyond periodic driving, considering the case of uniformly continuous and bounded 
rates. 



II. DEFINITIONS AND SETUP 



Let (Xt)t>o be a continuous time Markov chain on the probability space (Cl,J-,P), where 
0, = D(0,oo) is the space of cadlag paths (i.e., right-continuous with left limits) over a finite 
state space, which, without loss of generality, we take to be {1,2, .. . ,N}, and P is the Markov 
path measure with initial distribution tt. The process generates the right-continuous filtration 
J~t = C\ s>t cr(X r ,0 < r < s), representing information known about the process up to and in- 
finitesimally beyond time t, with \ J 7 , and evolves according to transition rates kij(t), such that 
the probability of jumping from state i to j within a time window [t,t + r] equals kij(t) + o(r). 
These rates are assumed to be such that the infinitesimal generator A(t) = (kij(t))^j =1 is continuous 
Vt > 0, T-periodic, and irreducible for some t* £ [0,T], but whose adjacency graph, representing 
which states currently communicate, may otherwise change over time and become reducible. We 
further require that the rates satisfy the so-called dynamic reversibility [27] or ergodic consistency 
condition [9], whereby kij(t) > <^=^ kji(t) > 0, which again ensures that the time-reversal of 
any realizable trajectory in the forward process is realizable in the backward one. The law of the 
process, [J-(i,t) = P(u £ 17 : Xt(u>) = i), satisfies the forward Kolmogorov equation j2l] 



g-A- W , (i) 

with initial condition /u(-,0) = tt, where •* denotes the adjoint. We take (Vj)j>i to be the random 
jumping times of the process and a% = X n , with o"o = Xq. 

Let us now define the objects we will primarily be concerned with, the entropy production 



5(t ,t)=log^* L ' 
ar [o,t-t ] 

equal to the logarithmic Radon-Nikodym derivative between the forward and backward path mea- 
sures (to be discussed shortly), and the action functional 
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representing heat dissipation, where log(//(X fo ,to)//i(Xi,i)) equals the net difference in system 
entropy between times to an d t. (fx(X s , s) here should be interpreted as fj,(i, s)\i = x s ( LL) )-) Exponential 
factors representing holding times between jumps have been canceled on the RHS of Q (see Refs. 

ITT]); a full representation of the forward path density can be found in ( |30[ ) in Section V. 
For a comprehensive justification of the definitions used above for entropy production and heat 
dissipation, see Refs. [UJ [JS] [26j 123 E3] . 

P[t 0} t] above is our Markov path measure restricted to [to,t] and P£ t _ t i is the corresponding 
backward path measure, defined as follows. Let P^ t _ tQ j be a measure obtained from P[t 0; t] by 
setting its initial distribution /j,~(-,0) = fJ.(-,t) and its rates feZ-(s) = fcy(t — s) for < s < t — to, 
and let r{u) s = lim s /^„ s uj s > be the path-reversal transformation, with uj G ^ a sample path. We 
then define P^ t _ tQ ^ = P^t-t ] or ~ Note that r is involutive on $7, preserving the cadlag property of 
paths, and that the law of the backward process, fJ- B (-, s), defined for < s < t — to, is implicitly 
a function of t due to r. It satisfies the final condition fi B (-,t) = fi~(-,0) = /u(-,t). Note crucially 
that the backward process is not a Markov process with respect to the filtration (^" s )o<s<t- Indeed, 
/i B (-,0) = /i~(-,t), and so the smallest cr-algebra that the event {Xq = i} is measurable with 
respect to under this process is Tt- It is, however, Markovian with respect to the backward time 
filtration Q t = f] s<t cr(X r , s < r). 

Reversing both the path and the rates in the backward path measure ensures that the transi- 
tions of a reversed path in the P~-governed process occur under the same local conditions (i.e., 
instantaneous rates) as those of the original path do in the forward process. Intuitively, if P[o,i]( w ) is 
the probability of observing the trajectory oj unfold as time runs forward from to t, then PS Aoj) 
is the probability of observing that trajectory unfold in reverse as time runs backward from t to 0, 
with the filtration Qt representing the past. From an Archimedian perspective "outside of time" 
(28], neither direction of time should be preferred a priori. S(0,t) is therefore truly a measure 
of irreversibility: Thinking of u as a spacetime curve instead of an oriented trajectory, 5(0, t) (a;) 
gives the log likelihood of uj being realized in a universe where time runs forward as opposed one 
in which time runs backward, with the sign of its expectation - dependent on which measure we 
integrate with respect to - indicating the overall direction of time's arrow. 

It has been argued recently [ID], we should note, that the expectation of S(0,t), the mutual 
entropy Ti(P[o tt ], P^ t i), is too sensitive to rare irreversible events to be a useful measure of irre- 
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versibility, and that the quantity A = \H{P m , \{P m +P$ jt] )) + §W(Pg t] , \{P[o,t] + P [o,t])) should 
be used instead. The "asymmetry" A equals the amount of information gained about the direction 
of time's arrow from watching one realization of the process. 

The first theorem that we will prove is the existence and differentiability of the free energy 
function of the action functional W(0,t), 



c w {\) = lim - logE^e^ '*)), (3) 

which guarantees a large deviation property for W(0, t)/t with rate function Iw( z ) = su PAeiR{^ z ~~ 
cw(\)}. Here K u t (') denotes expectation conditioned on the chain having distribution v at time 
to- Our second main result is the symmetry relation c^(A) = c^(— (1 + A)) between it and the 
free energy of the action functional under the backward process, 



4(A)= t hmilogE^(e^ S ^)), (4) 

which reduces to the usual Gallavotti-Cohen symmetry when the generator is a symmetric function 
of time within each driving period, in which case the backward path measure reduces to the 
path-reversed measure P[o,t] ° r considered in Refs. |18| \19\ 124"! [26] [27]. (Note that even in this 
scenario these authors' results do not apply, because the dynamics remain time-dependent and 
non-stationary.) See Ref. [32] for a discussion in the context of transient fluctuation theorems. 

In the definition (4), the expectation is taken with respect to P^ t y an d W (0,i) represents 
the heat dissipated by a process that traces the curve uj backward through time. Since each 
transition a\ — > o"j_i at time increases W B (0,t) by logfc - i0 - i _ 1 (ri)/fc (Ti _ 1(7 - j (Ti), we simply have 
W (0, t) = —W(0,t). Similarly, the entropy production under the backward process ^(O,*) = 
logdPg t JdF\o )t ] = -S(0,t). The key point is that while W and W B (resp. S and S B ) are 
distinct functions, when evaluated under P[o,t] an d P\ot]> respectively, they represent the same 
physical quantity. For additional discussion of this and the relation between forward and backward 
measures in general, see section 3.1 of Ref. |17j . 



III. LARGE DEVIATIONS OF HEAT DISSIPATION 



To prove the existence of the limit ([3|, we first derive a backward equation for the time- 
dependent moment generating function u\(to,t), whose elements u\(i,to,t) equal Ej j t [e AM/( -* '^], 
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where the expectation is conditioned on the process being at state i at time to- For notational 
economy we define A\(to,t) = e xw<yto,t ^ and w(i,j,t) = log(kij(t)/kji(t)). As a general rule we use 
A as a subscript in order to emphasize its distinction from time parameters, with the exception 
of the free energy. Finally, we use the convention that all matrix inequalities, denoted by :<, and 
limits are defined component-wise. 

Proposition 3.1: u x (tQ,t) satisfies the Kolmogorov backward equation 



dt t 



-u\{to,t) = -L x (t )u x (t ,t) 



(5) 



where 



L\{to)ij 



k ij (t ) 1+x k jl (t )-\ i + j 
-Ki(t ), i = j, 



(6) 



and Ki(to) = J2j^i (*o) *s ^ e escape rate from state i at time to- 
Proof: Because the jump times of a Markov chain are isolated almost surely and its paths 
are cadlag, we interpret the partial derivative in (j5j) to be the left derivative: ^u x (to,t) = 
lim^o+ " A ^ A ^" h,t \ Evaluating the second term in the numerator is the crux of the problem. 
We begin with 



lim u\(i,t - h,t) = lim E i to _ h [A x (t - h,t)} 

h— >0+ h^0+ 

= lim E iit() _ fc [E ii i _ fc (A A (to - h,t )Ax(t +,t)\F t0 )} 
h— >0+ 

= lim Eij _ h [A x (t - h,t )E iA ^ h (A x (t +,t)\F to )} 

h— >0+ 

= lim E itto ^ h [A x (t -h,t )E x to (A x (t +,t)}. 

h—>0+ u 

Here the limit to+ is taken before the limit h j 0, and in the last line we have used the Markov 
property inherited by A x (to,t) from Xt- Since to is n °t an accumulation point for jumps almost 
surely and we are operating in the small h limit, at most one jump can occur between to — h and 
to- The last line therefore becomes 
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lim E i|to _^ [ e ^(i^*o.*o) E XtQ}to (A A (t +, t))] 



2V 



lim ^ E i)t0 _, [ e M^ >*°) Extoj<o ( AA(to+) t) ) ^ .] 



/i->0+ 



2V 



lim ^ e A ^>'°) Ei )t0 _ A [E iit0 (A A (t +, t)) , 



/i->0+ 



3=1 



with 1,4 denoting the indicator of the event A. Regardless of whether a jump occurs at to (that is, 
Xt = j 7^ i)i no jumps may occur during a sufficiently small period after t$. The inner expectation 
therefore equals u\(j,to,t) and our limit is 



N 



lim + £ e Xw ^u x (j, to, m ht ^ h [l Xto=] \ 

3=1 

N 

= lim y2e Xw ^u x (j,t ,t)p(iJ,t -h,t ), 



(7) 



3=1 



where p(i,j,to — h,t) is the transition probability of being at j at time to given having been at i 
at to — h. For i ^ j this equals fey(to — h)h + o{h) and, for i = j, it equals 1 — iQ(to — h)h + o(h), 
with defined in the statement of the Proposition. We therefore have, by continuity of the kij, 



N 



lim u x {i,t -h,t) = lim V(%(t )^ + o{h))e Xu,{i ' j ' to) u x {j, to, t) + (1 - Ki(t )h + o{h))u x {i, t ,t) 

h— >0+ /i— >0+ ' — ' 

and the component-wise derivative becomes 



d 1 

— ux(i,t ,t)= lim -[ua(Mo,*) - ""A (Mo - M)] 
ato /i— >o+ n, 



TV 



^ %(t ) 1+A ^(to)" A uaO". *o, t) - Ki{t )u x {i, t , t) 



Collecting components into a vector equation, we obtain (|5]). □ 

Before stating the first main theorem, we introduce a concept from quantum field theory that 
will be useful. For a family of operators {0(s)} se R, define the time-ordering operator T [36] by 
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T Y\™ =0 O(sj) = 0(s n ) • ■ ■ 0(si)0(sq), where sq < • ■ ■ < s n is any finite sequence in R. Now, given 
to < ti, we define the time-ordered exponential 



"' x / h~ to n ,\ / ti — to , n , m \ / ti — to 



Tj^exp y = Jirn o exp^-^-^C>(s^_ 1 )j---exp^ 1 ^ u O(s?)jexp(^ x ^ OOff 

when the limit exists, where |s™ = to + — to)/w}" =0 is a sequence of uniform partitions of 

[to, ill- 
Recall that a matrix is called quasipositive if its entries are nonnegative, strictly on the diagonal. 
Our results rely on the fact that the time-ordered exponential operator, like the usual exponential 
operator, transforms quasipositivity into strict positivity. The proof requires some clever bounding 
arguments. 

Lemma 3.2: Let < to < t% and L : [0, oo) — ► Mn x n(M) be a continuous function whose values 
are quasipositive Vt > and irreducible for some t* G (to,ti). Then the matrix T(exp L(t)dt) 
has strictly positive entries. 

Proof: Without loss of generality, we take to = 0, the length of the interval being the only 
relevant factor. By uniform continuity of L{t) on [0,ti], choose sup tg j 0j f 1 ] i i<i<iv L(t)^ < M < oo, 
where the superscript denotes the negative part of the number, so that E(t) = L{t) + MI has 
strictly positive diagonal entries and nonnegative off-diagonal ones, Vt £ [0,ti]. Employing the 
partitions {s™} defined above, we have 



exp(|L(<_0Y--exp(|i^)) 

(t\ T(n v , Mti_ Mti_\ /ti. n . Mti 

^exp^^^^^.-exp^^)) 



Mti j. 

n 



e -M tl 



y e~^ (i + ■•■(/+ ^(^)) • (8) 

By irreducibility of L(t*), let < a < mini<j j<j^(E(t*) N )ij, and define E a to be the N xN matrix 
whose entries are all a. By continuity of E(t), choose 5 small enough so that Vt* < u\, . . . ,un < 
t* + <5, E{ui) ■ ■ ■ E{u N ) y E a . Finally, let k = \nt* /t{\ and I = [n(t* + 5)/h\ denote the index of 
the first partition point after t* and the last one before t* + 5, respectively, so that I — k > n8/t\ — 2. 
We can then rewrite (|8| as 
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e ~ Msi ( j + ^ (s - i} ) • • • ( j + ( j + ^ (s "0 • • • ( 

x(l + ^_ 1 ))-(l+^K)) 




Therefore, choosing < /? < e Mtl 5 N a/Nl, we have that the entries of T(exp J*^ 1 L(t)dt) are 
bounded below by j3. □ 

Note that by the irreducibility and ergodic consistency conditions on A(t), L\(t) is irreducible at 
t* , and, hence, the lemma applies to it. We recall that our assumptions on the generator A(t) are 
sufficiently weak that the rates kij(t) may drop down to zero, thereby changing the structure of 
its adjacency graph and possibly rendering it reducible. Irreducibility of A(t) over an arbitrarily 
small interval is sufficient, nevertheless, for all states of the chain to communicate after t*. 

Theorem 3.3: Given the periodicity, continuity, irreducibility and ergodic consistency condi- 
tions on the generator A{t), the free energy cw(X) exists, is continuously differentiable VA 6 M, 
and is independent o/vr. 

Proof: Rewriting the free energy as lim^oo \ log(7r,u\(0,t)), where (•) denotes the inner- 
product on M N , our strategy is to represent u\(0,t) in terms of the Floquet fundamental solution 
to ([5]), whose associated flow operator we will show has strictly positive entries and whose periodic 
component can be bounded so as not to affect the asymptotics. The Perron- Frobenius theorem 
will then allow us to identify the free energy as the principal Floquet eigenvalue of the fundamental 
solution, which depends smoothly on A. 

We begin by noticing that in our backward equation, u\(to,t) evolves according to the matrix 
— L\(to), whose off-diagonal terms are non-positive and whose corresponding flow will therefore 
not have exclusively positive entries. Instead of integrating forward, we therefore elect to change 
variables and integrate back to u\(0,t) from the known value u\(t,i) = (11--- 1)*. To wit, let 
r = t — to, and define u\(t, t) = u\(t — t, t) = u\(to,t) and similarly L\(r) = L\(to). Then 



13 



d d 

—ux(r,t) = -—u\(t ,t) = L\(t )u\(t ,t) = Lx(r)ux(r,t). (9) 

Recall that the fundamental solution to ([9| with initial condition u\(0,t) = (1 1 • • - 1)* is a 
square matrix whose N columns are N linearly independent solutions to the ODE. The Floquet 
theorem |41j guarantees that by periodicity of L\(t), this fundamental solution has the form 
$a(t, t) = e THx ^ Px(t, t), where Px is T-periodic and continuous in r and Hx(t) is complex. 
We choose in particular the principle fundamental solution, which satisfies <l>A(0,t) = I, so that 
ux(t, t) = &x( r ,t)ux(0,t). This theorem is derived by first noting that by periodicity of Lx, if 
$a(t, t) is a fundamental solution of ([9]), then so is $>\(t + T, t). This implies that the two matrices 
are linearly dependent, and hence there exists a nonsingular matrix C{t) such that $>\{t + T,t) = 
C(t)$ A (r,t). H x (t) is defined as ± log C(i), since nonsingular matrices always possess a logarithm. 
We may therefore identify e THx ^ as the flow operator that evolves solutions T units of time into 
the future, which is independent of the current time r. From ([9]), it can then be represented as 

e TH x (t) =T ^ exp J + L x (s)ds\ V0<r<t-T, (10) 

which, by the lemma, has strictly positive entries. A second dividend of this representation is that 
because the RHS does not depend on t, neither does Hx- 

By the Perron-Frobenius theorem, e THx possesses a nondegenerate positive, maximum eigen- 
value e Tl? ( A ) and a positive eigenvector vx spanning its corresponding one-dimensional eigenspace. 
This makes i?(A) the nondegenerate principal Floquet eigenvalue of the fundamental solution 
which, as a simple root of the characteristic equation of Hx, is continuously differentiable with 
respect to A by the implicit function theorem |23j. Evaluating the free energy function, 



c w (X)= lim ^logE 7r , (e w(0 '* ) ) 

t— >oo t 

= lim - log (tt, u a (0, t)) 

t—>oo t 

= lim -log(7r,UA(M)) 

t— »oo t 

lim -log(7r,e iHA P A (M)(ll---ir 



(11) 



t^oo t 

1 



lim - log ((tt, vx^Hvx, Px(t, t)(l 1 • • • 1)*)) 
0(A) + lim jlog(vx,Px(t,t)(ll ■■■!)*) 
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where we have used the uniqueness of the maximal eigenvalue and then the positivity of v\ to 
eliminate the dependence on ir. 

Our final task is to show the remaining limit vanishes by bounding the quadratic form Q(t) = 
(v\, P\(t, t)(l 1 ■ ■ ■ 1)*) between positive numbers, uniformly in t. To achieve this, we again exploit 
the time-ordered exponential representation of the flow. Taking t = mT + r, where < r < T, by 
the factoring property of such operators 



e mTH *e rH *P x {t,t) = r(expj L x (s)ds^j 

= T^expJ^ L\(s)ds^jT ^exp J L x (s)ds 



e mTH x 



T^exp J L\(s)ds 



Multiplying by e rHx e mTH ^ on the left, we have 



Px(t,t) = e- rw ^r^exp L x (s)ds . (12) 
The presence of r but not t on the RHS reveals that P\{t) = P\(t,t) is T-periodic, and the form 



of (12) ensures that it is uniformly continuous on its periodic domain. We may conclude from this 
that the matrix norm of P\(t,t) is bounded in t, and so the quadratic form is bounded above. 

Because we may have r < t* , the time at which the generator is irreducible, the lemma does not 
guarantee for us that T(exp J* Q r L\(s)ds) has strictly positive entries. However, it is straightforward 
to see by adding and subtracting M > sup se ^ r ^ 1<i<N Lxis)^ times the identity from L\(s) that 
this matrix exponential does have nonnegative entries, strictly on the diagonal. From this we may 
conclude that 



wx = T(exp J L x {s)ds\(l 1 • • • 1)* h 



with w\ 7^ 0. Taking R v ± to be the projection operator onto the orthogonal subspace of v\, by 
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Q(t) = (v x ,P x (t,t)(ll---l)*) 

= (v x , e~ rH >T fexp j* L x (s)ds\ (1 1 • • • 1)*) 

= {v x ,e~ rHx w x ) 

= (v x ,e~ rHx ((v x ,w x )v x + R v ±w x )) 
= e- r ^(v x ,w x } 2 >0. 

This is our uniform lower bound. □ 

Remark 3.4: One novelty of the proof of Theorem 3.3 vis-a-vis the proofs of large deviation 
principles by Lebowitz and Spohn [23], Jiang et. al. [19] and others is that it does not rely on the 
existence of an ergodic distribution for the dynamics. Indeed, no stationary distribution exists. By 
([!]), /u(-,t) evolves according to the periodic matrix A*(t), and so by the Floquet theorem can be 
represented as [i(-,t) = e tB P(t)7r, where P (t) is T-periodic. Because A*(t) is quasipositive always 
and irreducible at t* , by Lemma 3.2 and an argument analogous to that in the proof of Theorem 
3.3, e tB has strictly positive entries and thus a Perron-Frobenius eigenvalue. This eigenvalue must 
be 1, otherwise probability conservation would be violated. Taking v to be the corresponding 
positive eigenvector and R v ± the projection onto the subspace of M. N orthogonal to it, 

H(-,t) = e tB P(t)7r = (P(t)vr, v)v + e tB R v ±P(t)ir ~ (P(t)ir, v)v. (13) 

We see that the asymptotic limit of any initial distribution ir is T-periodic through P(t), and 
therefore that no stationary distribution exists. As proved in Theorem 5.2, this asymptotic limit 
takes the place of the usual stationary distribution in the expression for the instantaneous entropy 
production rate. 

Corollary 3.5: The time-averaged action functional W(0,t)/t satisfies a large deviation prin- 
ciple with continuous and strictly convex good rate function 

Iw(z) = sup{Az — cvk(A)}. (14) 

Proof: Existence and differentiability of cw(A) everywhere imply, by the Gartner- Ellis theorem 
[5], that W(0,t)/t satisfies a large deviation principle with rate function Iw(z) = sup AgK {Az — 
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ciy(A)}. Strict convexity of Iw( z ) can be deduced by contradiction. If the convex envelope of 
Iw( z ) (the supremum of all convex functions minorizing it) contained an interval of strict linearity 
with slope Ao, then the free energy's derivative would jump at Ao by Legendre duality, violating 
its continuity (see Ref. [38] for an intuitive discussion). 



Continuity requires a more detailed argument. Differentiability of cv^(A) implies that (14) 
can be computed from calculus: I\y(z) = X z z — cw(X z ), where X z is defined implicitly through 
c' w {\ z ) = z. As the free energy may not be strictly convex, however, A 2 is not necessarily unique. 
The set {X z 6 E : c' w (X z ) = z} is in fact a closed interval [A™ m , A™ aa; ], which reduces to a single 



point only for z corresponding to non-linear portions of c^(A). The RHS of (14) achieves its 
maximum at every \ z in this interval, including, in particular, at the endpoints. 

Let us now consider a sequence z n converging to z. We then have liminf n^oo X Zn > A™ n and 
limsup n _ +00 X Zn < X™ ax , which, by continuity of cw(X), implies that 



limsup%(z n ) = limsupA 2n z n - c w {X Zn ) < I\y{z) 

n^oo n— >oo 

< liminf \ Zn z n - c w (X z J = liminf I w (z n ), 

n— >oo n^oo 

or lim^oc I w (z n ) = I\v(z). 

Finally, recall that Iw{ z ) is called a good rate function if all sets of the form I\y{z) < a} 

are compact. This is implied directly by convexity and continuity. □ 

Remark 3.6: Differentiability of the free energy is crucial in the proof of the corollary. Without 
it, I\v(z) may be non-convex, with the Legendre- Fenchel transform of cw (A) yielding only its convex 
envelope. In such situations there exists no general method for accessing the rate function. This 
breakdown in Legendre duality has been proved [7] to be a necessary and sufficient condition for the 
nonequivalence of the canonical and microcanonical ensembles in statistical mechanics, whereby 
mean energies equal to slopes not present in the convex free energy profile (due to a nonanalyticity) 
cannot be realized in the canonical ensemble. Even given continuous differetiability of the free 
energy, however, we cannot improve upon the continuity result for the rate function to conclude 
differentiability. The reason is that the free energy may contain linear parts, each of which creates 
a nonanalyticity in the rate function. This is why we speak of the two as being related by the 
Legendre-Fenchel transform instead of the better known Legendre transform, which is defined only 
between differentiable functions. 

We now prove our primary result for the action functional. 
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Theorem 3.7: The free energy c^(A) exists, is continuously differentiable VA G M and satisfies, 
together with cw(A), t/ie symmetry relation 



cw(X) = 4(-(l + A)), (15) 

implying the fluctuation theorem 

I w (z) - I^(-z) = -z, (16) 

where 1^ is the continuous and strictly convex good rate function of the action functional under 
the backward process. 

Proof: Obtaining an explicit representation of the free energy c^(A) is simply a matter of 
bootstrapping from the work we did in Proposition 3.1 and Theorem 3.3. It is implied by the 
discussion in Section II that if we reverse time and the orientation of the driving rates kij{t), then 
the forward process becomes the backward process up to boundary conditions. In particular, the 
backward action functional W B (to, t)(u)) equals the heat accumulated by traversing the path co 
backward from time t — to to time 0. This immediately implies 



— uf(t , t) = -L x (t - t ) u B (t , t) 

Oto 

for the moment generating function u B (to,t), with components u B (i,to,t) = K B to [e^ wB ^ to,t ^]. Intu- 
itively, increasing to still decreases the moment generating function, only in the backward process 
the increments that are lost correspond to jumps at t — to- Were we to recapitulate the proof 
of Proposition 3.1, the filtration Tt used in the conditional expectations would be replaced by 
Qt, representing equivalently the future of the forward process and the past of the backward one. 
Making the change of variables r = t — to, with uf(r, t) = uf(to, t), we obtain 



JUf(T,t)=L A (r)uf(r,t). (17) 

The solution to this equation can be represented as uf (r,t) = e rH A p* (r,t)(l 1 • • • 1)* for some 
T-periodic P? and real H\, whose matrix exponential has strictly positive entries. Writing the 



free energy as c^-(A) = lim^oo j log(/i(-, t), u^{t, t)), by a chain of equalities similar to (11) and 
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the ensuing argument, c^(A) equals the continuously differentiable Perron-Frobenius eigenvalue of 
H?. The desired properties for its rate function Iyy( z ) then follow from the arguments in Corollary 
3.5. 

We now use the symmetry relation L% 1+A ) = L x (and L% 1+A ) = L\) observed from (6) to 
prove that 



(i?f (1+A) r = h x . (is) 



Indeed, as e TH * is a flow operator, mapping solutions of (17) T units of time forward (in the 



direction of increasing r), irrespective of the current time, it has the representation 



T^expj L x (s)ds^j, V0<r<t-T. 



Fixing one such t and letting {s"}o<j< n be the sequence of uniform partitions of [r, r + T] with 
mesh size T/n, we then have 



= i™o | ex p(^ L -(i+A)«-i)) • • •exp^L_ (1+A) K)^exp^L_ (1+A) (^)^ | 

= lim exp(-L x (s%))exp(-Lx{s%)\--exp(-L x (sl_ 1 ] 
n-»oo yn J \n J \n 

= lim exp( -L x (t-s%) )exp( -L A (t - s?) Y--exp( -L A (i - s™^) 
n— »oo yn J \n J \n 

= T[ exp / L x (s)ds ) 

V Jt-(r+T) J 



e TH \ 



where the final equality is justified by the fact that (t — r) — [t — (r + T)] = T. We were also 
able to interchange the limit and adjoint operators because a sequence of matrices A n converges 



component-wise to A iff A* n converges to A*. By the uniqueness of matrix logarithms, (18) holds. 



To prove (15), let H\v x = c\y{tyv x , where cvk(A) is the maximal eigenvalue of H x and v x 



is its corresponding positive eigenvector. By similarity of matrices to their adjoints, cy[/(A) is 



also the maximal eigenvalue of H x and, hence, of i?_/ 1+A \, via (18). But we have already shown 
c w(~ (1 + ^)) t° be maximal for H B ^ l+x y and so by uniqueness of the Perron Frobenius eigenvalue, 
c w (X) = 4 r (-(l + X)). 
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The fluctuation theorem ( 16 ) now follows from the Gartner-Ellis theorem: 



I\v{z) = sup{Az — cw/(A)} 

= sup{Az -<&(-(! + A))} 



(19) 



sup{-(l + A)z-c^(A)} 



= I^{-z)-z. □ 

Remark 3.8: The careful reader will notice that the A dependence in the relation (15) that we 



obtain matches that in the free energy symmetries of Refs. pjj] and [16], but not Ref. [21]. The rea- 
son is one of convention. The scaled cumulant generating function e(A) = lim^oo — \ log(e^ xw ^) 
in Ref. [23] satisfying the symmetry e(A) = e(l — A) is the canonical free energy function [38 , 
familiar from statistical mechanics when t is the particle number, W{t) the Hamiltonian of a con- 
figuration and A the inverse temperature, but distinct from our free energy function Q by the 
minus signs. It nevertheless gives rise to the same symmetry for the rate function as ours (in the 
special case of homogeneous dynamics) via the Legendre-like identity Iw( z ) = su PAeK{ e (^) ~~ 
|24j . For an in depth discussion of the relationship between the physics and mathematical notions 
of entropy and free energy, see Ref. [38J . 

Remark 3.9: When the generator A(t) is symmetric within each driving period, 1^ = Iw 



and our fluctuation theorem (16) reduces to the usual I\y{ z ) — Iw{~ z ) = — z , which says that 
the odd part of I\y is linear with slope —1/2. When time-inhomogeneities are involved, however, 
the fluctuation theorem no longer represents an internal symmetry of a single rate function. If 
we interpret it as a formula for computing the large deviations of heat dissipation in the forward 



process in terms of those in the backward process, then the invariance of (16) with respect to the 
joint transformation ify *-* I\y, z «-> —z, which amounts to reversing the roles of the forward and 
backward process (recall that W B (0,t) = -W(0,t)), indicates an invariance in the large deviations 
of heat dissipation with respect to time reversal. 



IV. LARGE DEVIATIONS OF ENTROPY PRODUCTION 

We now prove results analogous to the ones above, but with the action functional W(0, t) 
replaced by the entropy production 5(0, t) of the stochastic particle, whose free energies under the 
forward and backward processes are 
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c s (X) = lim - t logE^ (e A5(0 ' 4) ) and cf (A) = hm - f logE^.^^e^^) . (20) 
Because the proofs are very similar, we cover only the modifications that must be made. 

Theorem 4.1: Given the periodicity, continuity, irreducibility and ergodic consistency condi- 
tions on A{i), the free energy 05(A) exists, is continuously differentiable VA 6 M., and is independent 
of it. The time-averaged entropy production S(0,t)/t therefore satisfies a large deviation principle 
with continuous and strictly convex rate function Is(z) = supxeE.{^ z ~ c s(^)}- 

Proof: Let v\(to,t) denote the moment generating function of S(to,t), with elements 
v\(i,to,t) = ^i,to[ e ^ ^ ]• The derivation of the backward equation we prove for v\ begins iden- 
tically to that in Proposition 3.1, except that we replace w(i,j,t) with 

»(*, 3, to, t) = log / + log 7 . 

These new increments incorporate the boundary terms so that the entropy production becomes an 
additive process like the action functional, with the representation 



S(t ,t) = Yim^2s(X u ,X u+1 ,ti,t i+1 ), 

i=0 

where V = {tj}™ =0 is a partition of [io>i]- Starting from the expression ffty, we have 



N 



lim v x (i,t -h,t)= lim Te^ t0 - h Mv x (j,t ,t) P (iJ,t -h,t ) 

h— >0+ h^0+ ' — ' 



By the forward evolution equation ([TJ for 



fi(i,t ~h)\ X dtn(i,to) , . 
r- — \ — = 1 — hX — - — r — h o(h) 

A Kj> to) 



1 - hX E%(*o)^°( + ^(to) + o(M, 
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which we insert into the previous expression to obtain 



A' 



lim v\(i, to — h, t) = hm > 



%(t - h)fi(i,t - h) 
kji(t - h)fi(j, t ) 



{kij(t )h + o(h))v x (j, t , t) 



+ (1 - Ki(t )h + o(h))v x (i,to,t) 



v\(i,to,t)h\ kji(t ) 7t~7~~t + h\Ki(to)v x {i,to,t). 



Our component-wise derivative for v\(i,to,t) then becomes 



1 

— v\(i,t ,t) = lim -[t> A (Mo,0 - ^a(Mo - M)] 
ato /i— >o+ n 



E %(to) 1+A ^(t )- A to, t) 



N 



+ ( a e %(*o)4 z t4 + ^ - ) v ^ to ^ 



H(i,t ) 



which yields the linear backward equation ^-v\(to,t) = —M\(to)v\(to,t), where 



(21) 



M x {t ), 



%(to) 1+A ^(to)- A (gM) A , 



[AE^M*o)gfg + (l- A)^(to), i = j 



(22) 



The existence of the off-diagonal terms of M\(to) for to < appears problematic here, because 
we are dividing by fj,(j,to), which may be zero. However, in order for this to be a problem, we 
must have kij(to) and n(i,to) both positive. By continuity of these quantities, they must have been 
positive on [to — S, to] for some small 5, ensuring that to) > 0. The same argument justifies the 
fmiteness of the diagonal terms. 

The rest of the proof is identical to that of Theorem 3.3 and Corollary 3.5, where we make the 
change of variables r = t — to and use the Floquet theorem to factor the new solution v\(t, t) into 
a periodic matrix and an exponential flow matrix with strictly positive entries. The free energy 
cs(A) is then the principal Floquet eigenvalue of the latter, which by uniqueness is a continuously 
differentiable function of A, implying a large deviation principle with continuous and strictly convex 
good rate function equal to the Legendre-Fenchel transform of 05(A). □ 
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Remark 4.2: Equation (21) and definition (22) reduce to equation (11) and the subsequent 



definition of £(X) in Ref. [19] when the time-dependence of our transition rates is dropped. Con- 
spicuous in the definitions of M\{to) and £(X) is that, unlike for L\, the adjoint and A — * — (1 + A) 
transformations are not inverses, implying that the analogue of H\ in the proof above - call it 



H\ - will not satisfy (18). This does not invalidate the fluctuation theorem, however, because we 
merely require that H\ and .ff*/ 1+ ^ have the same maximum eigenvalue, not necessarily be equal 
themselves. The fluctuation theorem for entropy production, in fact, is very straightforward. 

Theorem 4.3: The free energy c<?(A) exists, is continuously differ entiable VA £ R and satisfies, 
together with 05(A), the symmetry relation 



c 5 (A) = cf(-(l + A)), 

implying the fluctuation theorem 

l s (z)-l§(-z) = -z, (23) 

where Ig is the continuous and strictly convex good rate function of the entropy production under 
the backward process. 

Proof: P[o t t] an d PS ^ are mutually absolutely continuous by the ergodic consistency of A(t) 
|15j . That their Radon- Nikodym derivatives with respect to each other are then reciprocals almost 
surely is a basic measure-theoretic fact [11]. Together, these justify the moment generating function 
symmetry 




which is equivalent to the fundamental fluctuation relation of Harris and Schiitz [17], cast instead 
in a quantum Hamiltonian formalism. We then immediately have the free energy symmetry 
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cs(X) = hm \ logE 7ri o(e AS (°'*))= ttn \ bgE* ^(e-^+W ^ cf (-(1 + A)), 

in which the existence and continuous differentiability of the RHS comes along for free, along 
with the stated properties of its corresponding rate function. By the Gartner-Ellis theorem and 



equalities analogous to (19), this implies the fluctuation theorem symmetry. □ 



Informally, the fluctuation theorem symmetry (23) yields the following generalization of the second 
law of thermodynamics: 



P(S(0,t)/t = z) 



~ e 



zt 



(24) 



P B (S B (0,t)/t = ■ 

This has the interpretation that spontaneous positive fluctuations in the time-averaged entropy 
production of the forward process are exponentially more likely than spontaneous negative fluc- 
tuations in the backward process, and vice-versa. Thus, for processes driven by time-dependent 
protocols, the distributions of entropy production fluctuations in universes in which time moves 
forward versus backward are interrelated. Only when the protocol is symmetric within each driving 



period (i.e., has no temporal orientation) does (24) reclaim its usual irreversibility interpretation 



that positive entropy production fluctuations in the forward process are exponentially more likely 
than negative ones. 



Taking Az = [z — e, z + e] , — Az = [—z — e, — z + e] and \Az\ = e, we formalize (24) as follows. 
Corollary 4.4: 

| i^O & 1 ^ PB(S S B%T/tT-L) = *' 

Proof: By both the large deviation upper and lower bounds [6J, lim^oo j\ogP(S(0,t)/t G 
J) = —'mf^jIs(C) for any closed, bounded interval Jcl, and similarly for S B (0,t). Therefore, 
by (23) and continuity of Is and I B , 



lim lim 1 log „£S f) w eAZ l x = lim -mf/s(C)+ mf I§(Q 



-J S (*) +/#(-*) 
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V. TIME- AVERAGED ENTROPY PRODUCTION AND ENTROPY PRODUCTION 

RATE 

Our final main result is a derivation of the asymptotic time-averaged entropy production and 
its associated instantaneous entropy production rate. We begin with the following lemma, which 
is a well-known large deviations result for countable sequences of random variables [B] and requires 
only slightly more work in the continous parameter case. 

Lemma 5.1: The time-averaged entropy production 5(0, t)/t converges exponentially and P[o,t] - 
a.s. to c' 5 (0). 

Proof: By Theorem 4.1, the rate function 



I s (z) = sup{Xz - c s {\)} 



(25) 



is strictly convex. It is immediate from the definition (20) that cs(0) = 0, and so from the inverse 
transform cs(X) = sup zeR {Xz — Is(z)} we obtain 



= c s (0) = sup{-/ s (z)} = - inf {I S (z)}, 

which, along with strict convexity, implies that Is(z) attains its minimum and zero uniquely at 
some z* . That z* = c' s (0) can be seen from the following argument. Recall from the proof of 
Corollary 3.5 that Is{ z ) = X z z — cs(X z ), where A 2 is defined implicitly through c' s (X z ) = z. At the 
same time, since cs(0) = 0, it is clear that (25) is minimized when A = X z * = 0, or z* = c' s (0). 



Exponential convergence of 5(0, t)/t to c' s (0) is now immediate, because for any e > 0, the large 
deviation upper bound 



limsup-logP[ 0i4 ] 



guarantees that for large t, 



5(0, t) 



4(o) 



> e/2 < - inf I s (z) 

|*-c' s (0)|>6/2 



-a e < 



Pi 



[o,t] 



5(0, t) 



t 



4(0) 



> e/2 < e 



(26) 



(The factor 1/2 multiplying e will be needed in a triangle inequality argument to follow.) 
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Almost sure convergence now follows from several applications of the Borel Cantelli lemma, 
which says that if the probabilities of a countable sequence of events are summable, then the 
events happen finitely often almost surely (i.e., only a finite number of them occur). In this case, 
we have that the events 15(0, n)/n—c' s (0)\ > e/2 occur finitely often, where time has been restricted 
to the positive integers. To show that the same holds for general t, let (ij)j>i be any sequence of 
real numbers tending to oo. Then 



[o,t] 



5(0,^) 5(0, L^j) 



> e/2 



< P 



[0,t] 



< p 



[0,t] 



S(0,tj) 



4(0) 

4(0) 



+ 



4(0) 



5(0, [t,\) 



[tj 



> e/2 



>e/4 \+P [0 ,t] 



5(0, L%J) 



[tj 



4(o) 



>c/4 ■ 



But by an inequality analogous to (26), these latter probabilities can be bounded by a common 



exponential for large tj, indicating, by Borel-Cantelli, that S(0,tj)/tj and 5(0, [tj\)/[tj\ differ by 
more than e/2 only finitely often (almost surely). The triangle inequality then gives us |5(0, tj) /tj — 
4(0)1 < e f° r large t, precluding, as the tj's were arbitrary, the possibility of |£(0, t)/t — 4(0)1 > 6 
for large t. This is precisely the definition of almost sure convergence to c- s (0). □ 

Theorem 5.2: The time- averaged entropy production satisfies 



< lim 



5(0, t) 1 



/ e p (s)ds < oo, 
Jo 



t—>oo t T 

where the convergence is exponential and P[ 0t ^-a.s., the instantaneous entropy production rate 



(27) 



is defined for < s < t, and v(-,t) = limt_ i . oc fi(-,t) is T -periodic. 

Proof: By the lemma, we already have convergence in both senses to c' s (0) , so what remains 
is to evaluate this derivative. In doing so, we make use of the fact that the free energy is convex 
and that limits and derivatives commute for sequences of convex functions [6] . 
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C5 '(0) = ^lim^logE^ (e A ^)) 
oA *-*oo t v ' 

= lim—log^o^ '*)) 

t^oo t OA ' 

= lim ^E 7rj0 S'(0,t) 



A=0 



A=0 



Recall that TL(P[ j], -P^ t i) here is the relative entropy of P[o,t] with respect to Pp t p which are 
mutually absolutely continuous by the ergodic consistency criterion on A. By nonnegativity of the 



relative entropy, we have the left inequality of (27). Our goal is now to factor the Radon- Nikodym 
derivative dP^^/dPS^ for finite t. To accomplish this, we note that on the event that the Markov 
chain jumps m times between times to and t, its path to is characterized by the states (o"i(o>))|^ 
it visits in sequence and the waiting times (Tj+i(u;) — Tj(u;))™Q between them. (Here we have set 
To(w) = to an d r m +i(a>) = t.) Because the former are discrete and the latter have a density, as well 
as the fact that the number of jumps J{oS) = J([to,t])(u) over the interval is finite a.s., the path 
measure P\t £ has density 



f[to,t]( U ) = /[to,t],J( w )(o"o(w), • • • , <7j(w)(w),Tl(w), • • ■ , Tj {oj) (uj)) 



(29) 



where 



f[t ,t],m{^0, ■ ■ ■ ,<rm,n, . . . ,T m ) = P{J{[t Q ,t]) = m)fj,(a ,t Q ) X 

TO— 1 



n k 



i=0 



«-CTi(Tj+i)exp 



K rJi (s)ds 



exp 



K am (s)ds 



(30) 



A similar density /r tot i(^) exists for the measure P[ iot i- For notational clarity in what follows, 
let us write the restriction of a path u to an interval £ C t as we. Given an M E Z and 



£ > 0, define the partition variables = sf 1,1 = || and scale parameter /i 
P^ t j(cj) = Pj~ t j(r(u;)), we have by the Markov property 



±_ 

M ■ 



Recalling that 
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]n(P m ,P { l t] ) = i^l O g^gi(a;)P [0)t] (da;) 



[o,t] 

1 f ! / M w 0,0) ^ /( S£ , Sf+1 ](w( S£lS£+1 ]|X S£ = W S£ ) \ 

Un^U-frMo-O) 11 /,^ 1 iW-W 1 ,|Jf«=r(„) M ); , ^ ( ' fc ' ) 

ly / iogf mK.^/c,.,,,,!^,,,.,,,,]!^.'^) ) P L . t 



1=0 



n Vm(^ + i,^+i)/ (sm _ 1 _ £)Sm _, ] (^M( Sm _ 1 _ <)Sm _,]|^ m _i^ = K^)s M -i-J 



Here we have rewritten the boundary term ^-^J^ q) = ^(^ft) as ^ ne telescoping product 
rf^ln 1 ^ Wa i ,Sl ^ anc j reversed the order of the density terms in the denominator. On the sets 

llt-V ti(uj Se+1 ,s e+1 ) J 



^i,t,M,i,j = £ £1 u)g e = uj se - = i and to Sl+1 = w S£+1 - = j}, 

which partition 0, up to a set of zero measure (the sg are continuity points of Xt a.s.), the densities 
on top and botton and the measure integrated against simplify so that we obtain 



l^ 1 ^ f n{i,s e )[P{X Sl+1 =j\X Se =i)+o{h)] 
* h h k,u,, " M se +1 )[P~(X SM _ e = = J) + o(h)] 



=0 i,j=l Jil £,t,M,i,j 
Af-1 N 



M-l . 2V 



V v V log— tt^t ^(n{i,s e )k ij (s e )h + o(h)) 

7 V & V M ^ log— — ^ r. 



The o(h) correction in the first line is justified by taking t J. to in ( 30 ) and noting that P(J([to, to + 



m) = o{h m ). Letting M f oo now in the Riemann sum, by continuity of fj,(-, t) and the rates, 



By Remark 3.4, for all initial distributions ir, fJ>(-,t) is asymptotic to a periodic distribution 



v(-,t) equal to the RHS of (13), with which we may replace it in the expression above in the limit 
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t — > oo. Let us now take 0(s) to be the integrand on the RHS of (31 ), so that <p(s) ~ e p (s). Given 



e > and choosing an M large enough so that \(f>(s) — e p (s)| < | for s > M, for sufficiently large t 



If 1 1 /" M 1 f* e 

- \(j)(s) - e p {s)\ds < - / \(j){s) - e p {s)\ds + - -ds 
1 Jo 1 Jo 1 Jm z 

e e ft-M\ 

We may then finally conclude, by periodicity of e p (s), that 



1 /"* 1 /' T 

= lim - / e p (s)ds = — / e p (s)ds. □ 
* Jo J 7o 



Remark 5.3: The result (27) that we obtain above is the continuous time analogue of Ge 
et. al.'s complete entropy production rate [16] for discrete time, periodically time-inhomogeneous 
Markov chains. The form of e p {s) itself is a time-dependent generalization of both (33) in Ref. 
|14j . where e p {s) is interpreted as the difference between backward and forward dynamical entropy 
rates, and (4) in Ref. [19]. From a dynamical systems perspective, v(--,t) is an attracting limit 
cycle for fJ>(-,t) in the space of distributions on {1, . . . , N}, representing a sort of periodic steady 
state. The mean entropy produced along this cycle is the mean entropy produced by fJ>(-,t) in the 
long time limit. We see below that when the time dependence of the transition rates is dropped, 
the limit cycle collapses to a fixed point and e p (s) reduces to the constant entropy production rate 
of homogeneous chains. 

Corollary 5.4: When fey(t) = k{j, VO <i,j< N, 



where the convergence is exponential and P^^- almost sure and \x is the unique invariant distribution 
ofX t . 

Proof: The existence and uniqueness of [i are guaranteed by the ergodic theorem. We then 
have t) = limt^oo //(•, t) = (J,, from which the result immediately follows by Theorem 5.2. □ 
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VI. BEYOND PERIODIC DRIVING 

The key modeling assumption made in this paper is that the transition rates driving the Markov 
chain are time-periodic. Despite the wealth of interesting physical and biological processes charac- 
terized by this type of driving, examples of which were given in the introduction, an open question 
remains as to whether there exists a reasonable set of minimal conditions on the transition rates 
that are weaker than periodicity but still guarentee existence of an AFT. By "reasonable", we 
mean that the conditions be both concise and easily verifiable for a wide range of applications. It 
is well known that transient fluctuation theorems make no assumptions about the protocol driving 
the process, suggesting that the minimal conditions necessary for an AFT to hold truly would be 
minimal. An initial guess might be that uniform continuity and boundedness are sufficient, which 
would prevent the number of jumps of the chain from growing faster than linearly in time as well 
as preventing pathological behavior due to discontinuous or infinitely rapid driving. The following 
result shows that this guess is incorrect. 

Proposition 6.1: Given arbitrary constants < a < [3, there exists a Markov chain whose 
transition rates are uniformly continuous and bounded between a and (3, but whose time-averaged 
entropy production does not satisfy an AFT. 

Proof: The plan is to take < a < (3 as given and then construct a chain with the properties 
above. We begin by choosing distinct constants a < kij < (3 for 1 < i, j < N such that the 
generator A c = (kij) does not satisfy detailed balance, which of course is always possible. Note 
that A c is irreducible because all its entries are positive, and so it possesses a unique ergodic 
distribution. We now choose a number 7 > 1 such that a < 7 fey < (3 for all i and j, and let t be 
the mixing time of the chain generated by 7vl c (the typical time until convergence to its ergodic 
distribution), which is equal to the reciprocal of the second largest real part of the eigenvalues of 
^A c . The generator A(t) of our chain is constructed as follows. With to = 0, we choose t\ S> r 
and then iteratively set tk = ktk—l- A(t) is initially defined to be A c for i2fc < t < ^2fc+i and ^A c 
for t2k+i < t < t2k- We then modify it by smoothing out the discontinuities in any manner that 
leaves A(t) uniformly continuous in time. 

In order to prove that S(0,t)/t does not satisfy a large deviation property and, hence, an 
AFT, we show that its free energy cs(A) fails to converge almost everywhere. To begin, define 
cg(\,t) = j logE 7I - j o(e A5 (°'^) so that 05(A) = lim^oo cs(A, t). Now let e p denote the instantaneous 



entropy production rate of a process generated by A c , which is defined by (32 ) and must be positive 
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because the rates kij do not satisfy detailed balance. Note that e p is homogeneous of degree 1 with 
respect to the rates, so that the entropy production rate of the process generated by jA c is je p . 
Having defined the tf~ to grow such that ifc-i/ifc — ► 0, over timescales much longer than the mixing 
times for A c and ^yA c , we see by Corollary 5.4 that 

{te p + o(l), t 2k <t < t 2k+ \ 
tje p + o(l), t 2 k+i < t < t 2 k, 
where o(l) denotes a term that vanishes as t — > oo. This implies that for A > 0, 



limmfc 5 (A,t) = lim - logE^ (e A(tep+o(1)) ) 

t^co t^co t 

= lim \ logE^ (e A ^) 

< lim - logE^ (e A( * 7ep+o(1)) ) 

t-^oo t 

= limsupc5'(A, t) 

t^oo 

and hence c(A) does not exist. The inequality is reversed for A < 0, so we see that the free energy 
only exists for the trivial value A = 0. □ 

Since uniform continuity and boundedness are implied by continuity and periodicity, the preceding 
result shows that the minimal conditions on the transition rates are closer to those assumed in this 
paper than one might initially suspect. What periodicity guarentees but uniform continuity and 
boundedness do not is that the rates cannot be tuned over arbitrarily long timescales, exactly the 
loophole we exploited above. It remains to be seen, however, how this condition can be formulated 
more precisely and whether there are other conditions must be included in our minimal set. 
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